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Abstract — In this paper, we develop an information entropy- 
based metric that represents the statistical quality of the gen- 
erated binary sequence in Truly Random Number Generators 
(TRNG). The metric can be used for the design and optimization 
of the TRNG circuits as well as the development of efficient 
post-processing units for recovering the degraded statistical 
characteristics of the signal due to process variations. 

I. Introduction 

A truly random number generator (TRNG) is a binary 
generator that is capable of producing independent and unbi- 
ased bits in a nondeterministic and irreproducible process (TJ. 
A TRNG is a critical part of cryptographic systems, where 
it provides the necessary uncertainty or entropy in order 
to secure the message transmission. Recently, discrete-time 
chaotic systems have been significantly used in the design of 
high speed TRNGs. Chaos is the non-predictive behavior of 
certain nonlinear dynamical systems that happens due to the 
high sensitivity to the initial state. Any small perturbation can 
grow exponentially in the system and can result in a non- 
predictive chaotic behavior 0. 

A discrete-time chaotic map, formed by the iteration of 
the signal in a simple nonlinear map function, can be used 
for the generation of truly random numbers. Piecewise linear 
input-output (I/O) characteristic curves have been extensively 
used as the map function for the generation of a binary 
output sequence according to the state of the piecewise linear 
function HI. The capability of integration, high speed, and the 
high quality of generated bits due to the simple discrete-time 
nature of the system make the discrete-time chaotic maps very 
good candidates for the generation of random numbers. 

In practice, the process variations in the circuit implemen- 
tation usually lead to a degradation in the statistical character- 
istics of the generated binary sequence. This necessitates the 
utilization of a post-processing unit to recover the statistical 
characteristics of the bit sequence. Von-Neumann [3], SHA- 
1 01 , and SHA-2 J51 are the most widely used post-processing 
algorithms. All of the post-processing algorithms should trade 
the bit-rate off with true randomness of the generated binary 
sequence. However, these algorithms often post-process the 
data blindly, i.e., without any knowledge of the non-ideal 
nature of the binary sequence. The development of methods 
that can predict the statistical characteristics of the binary 
sequence can enable the design of efficient post-processing 
units that can increase the efficiency of the TRNG. 



The true randomness of the generated binary sequence is 
traditionally validated against NIST 800-22 randomness test 
suite [61. NIST 800-22 is a pass/fail test that takes a large 
binary sequence as input and can only prove that a sequence 
is not random. This randomness test can not be used for the 
evaluation of the true randomness of the binary sequence. 
NIST 800-22 randomness test suite also does not provide the 
designer with much information regarding the efficiency of 
the design in order to optimize the performance of the TRNG 
circuits. 

In this paper, we propose a metric for the performance 
evaluation of the discrete-time chaos-based TRNG circuits. 
The information entropy of the generated binary sequence can 
represent the quality of the statistical characteristics of the 
generated bits. We develop a methodology to find the entropy 
of the generated bits according to the discrete-time chaotic 
map. 

II. Discrete-Time Chaotic Sources: Fundamentals 

Discrete-Time chaotic maps are formed by the iteration of 
the output signal in a transformation function M(x) : (0, 1) — > 
(0, 1) in a positive feedback loop as given by 



t-n+l 



M{x n ) = M n (x ) 



(1) 



In this equation, n represents the time step, xq is the initial 
output signal (initial state) of the system, and x n is the output 
signal at time step n. For example, for the sample map function 
shown in Figure \V[a), the transformation function is given by 



3v3 

M{x n ) = — — x„{l 



(2) 



As shown in Figure 0I a X the input and output ranges are both 
(0, 1). Note that (0, 1) is an arbitrary choice without loss of 
generality since any range can be scaled to (0, 1). 

For a discrete-time chaotic map, a density distribution 
function /„ (x) is defined as a function of the input parameter x 
and the time step n. f n (x) demonstrates the density of possible 
states in the system at time step n (after n iterations). A higher 
/„ (x) demonstrates a higher probability of being at x at time 
n. Since the output of the system at time n + 1 is only a 
function of the output of the system at time n, f n +i{x) can 
be obtained recursively from f n (x) using the Frobenius-Perron 
operator P, as given by J7) 



f n +i(x) = Pf n (x). 



(3) 




0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 



(b) 30 



25 



20 



f„(x) 15 



10 



0.2 0.4 0.6 0.f 

X 



Fig. 1 . (a) The example map. (b) The density distribution of the map. 

For example, in the sample map shown in Figure Q2 a X me 
density of states in Xb at time n + 1, f n +i(xb) is obtained as 
follows, xi, is the output value of the system for input values 
Xti and x t 2- The transformation of the density of states from 
x ti{2) t° x b is inversely proportional to the slope of the map 
as shown in Figure [TJ a). Therefore, f n+ x{xb) can be given by 



fn+l(xb) = 



1 



-Jnixtx) + 



1 



rfn(x t2 ), (4) 



\M'(x tl )\ JnK \M'{x t2 )[ 

where M'(x) is the derivative of the map function M(x) at 
point x. 

Under certain conditions, which are usually satisfied for 
TRNG maps, the density distribution asymptotically converges 
to a stable distribution foo(x) as n —> oo, regardless of the 
initial value of the system [8 |. In this case, foo(x) can be given 
by solving 

foo(x) =Pfoo(x). (5) 

The analytical solution of the asymptotic density distribu- 
tion is not straight-forward for an arbitrary map function due 
to the explicit definition of the Frobenius-Perron operator P. 
Therefore, we develop a simulation-based method to obtain 
the asymptotic density distribution of a discrete-time chaotic 



map. The ergodic nature of the problem enables the finding 
of the asymptotic density distribution by the iteration of an 
arbitrary initial point in the map function and following the 
output trajectory to find the state density distribution. However, 
this method needs to be carefully implemented since the digital 
nature of the computer can lead to periodic patterns in the 
output. We divide (0, 1) to L equal parts that are represented 
by to, th-\, where = i/L. We then store the values of the 
map function for the L selected values M(to), M(i£_i). 
The initial value of the map xo is randomly selected from 
to, ti_i. We define the iterative relation between input and 
output as 



x n+1 = yfloor[L x M(x n ) + a n ], 

Li 



(6) 



where a n is a random variable with uniform distribution in 
(—1, 1), and x n is the digital output value at time n. This I/O 
relation ensures that x n+ i g tx, •••,*£> which is the necessary 
condition for the iteration of the output in the system. The 
random variable a n is added to prevent the periodic patterns 
that can appear due to the digitization of the continuous map 
function. If L is chosen very large, a n will look like a noise 
in the order of 1/L. 

The first several iterations of the system need to be discarded 
in order to ensure that the output value has reached the 
asymptotic density distribution. In the next K iterations, we 
count the number of visits to each point f,-, and call it 
Num(ti). The density of states at point ti is given by 



In {ti 



L , . 

—Num(ti). 



(7) 



Note that increasing L and K will increase the resolution of 
the response, while requiring more simulation time. Moreover, 
since Num{ti) is an integer function, the resolution of the 
density function is limited to L/K. Therefore, it is necessary 
that K » L for an acceptable resolution. In Figure [Jib), the 
asymptotic density distribution of the sample map M(x) has 
been obtained by using the developed method. In this map, the 
very low slope of the map near x = Xf,, where M(xb) = 1, 
leads to high density of states near x = 1. This is due to the 
transformation of a large span in the neighborhood of Xb into a 
much smaller span in the neighborhood of 1. The high density 
of states near x = is due to the fact that the neighborhood of 
x = 1 is transformed to the neighborhood of x — 0. Therefore, 
a high density near x = 1 results in a high density near x = 0. 

III. Statistical Characteristics of the Generated 
Binary Sequence 

In order to generate a binary sequence from a discrete- 
time chaotic map, (0, 1) needs to be partitioned to two bit- 
generation partitions 5(0) and 5(1), which correspond to the 
bits and 1, respectively. Any of the partitions 5(0) or 5(1) 
can be in general non-connected. For example, in the sample 
map shown in Figure [TJ we assume that 5(0) = (0,Xb), and 
5(1) = (xb,l), where Xb — l/v3 is the abscissa of the 
maximum point. 



Output value x corresponds to (or 1), if and only if x € 
5(0) (or x e 5(1)). Next, we develop a methodology for the 
calculation of the information entropy of the output sequence, 
as a metric for the evaluation of the statistical characteristics 
of the generated binary sequence. 

A sequence of N bits, called an TV-bit sequence, has 
2 N possible cases according to the state of each bit. We 
find the partitions that can lead to each of the 2 N cases. 
Sjy(ii, ijy) is defined as the partition that includes all the 
initial state points x that will generate the TV-bit sequence of 
(ii, ...,ijv). where ii, ...,in & {0, 1}. For example, £3(0, 0, 1) 
is a partition of (0,1) that includes all the initial state points 
that will lead to the generation of the sequence (0,0,1) in 
the three iterations. Note that Si — 5 corresponds to the bit- 
generation partitions 5i(0) = S(0) and Si(l) = 5(1), which 
are defined by the bit-generation circuit design. All Sn for 
N > 2 can be found based on S and the map function M(x). 
Sn is related to S through the following relation 

xi e S N (ii, ...,ijv) Xj € S(ij), j = 1, —,N, (8) 

where ij e {0, 1}. We find the 2 N+1 partitions for an 
(N + l)-bit iteratively based on the 2 N partitions for an 
TV-bit sequence. Assume that Sn(ii, ijv) is known for 
il,...,iff € {0,1}. According to the equation ||8}, it can be 
shown that 

xi G S N (i!,...,i N ) O x 2 e S N -i(i 2 ,—,iN), (9) 

xi e 5jv+i(«i, in, in+i) »i2 6 S N (i 2 , in+i)- (10) 

Therefore, if Xi £ 5jv(h, n), then either x 2 E 
Sn(i2, — , in, 0) or x 2 £ Sn(i2, — , in, !)■ For each initial 
point xi, we find the output x 2 . Then, we find the correspond- 
ing partition of x 2 among all Sn since 2 N partitions of Sn 
are known. Equation (ITOb shows how this can lead to finding 
the corresponding partition of xi among all 2 N+1 partitions 
of Sn+i- 

In the next step, we find the probability of the occurrence 
of each of the 2 N cases in an TV-bit sequence. It is reason- 
able to assume that the output value follows the asymptotic 
density distribution since there is no access to the output 
value in the output binary sequence that can perturb the 
distribution. The occurrence probability of the case (ii, in) 
is called Pn(h, ijv), and is given by the integration of 
the asymptotic density distribution over the corresponding 
partition Sn(h, — >*at)> as given by 

Pv(ii, ...,izv) = / foo(x)dx, (11) 

J Sw(ii,...,ijv) 

where ii,...,2jv € {0,1}- For the case of N = 1, the 
probabilities Pi(0) = P(0) and -Pi(l) = -P(l) demonstrate 
the steady state probability of occurrence of and 1 in the 
system. The bias, b, in the binary sequence is defined by 

&=|P(0)-i| = |P(l)-i|. (12) 

The bias is the most important metric for the randomness of 
binary sequences. 



For example, in the sample map shown in Figure HI a), we 
have Si(0) = (0,x b ) and 5 X (1) = (x b , 1). Therefore, P(0) 
and -P(l) are given by 

P(0) = f " foo(x)dx, 
Jo 

P(l)= f f^dx, (13) 

where P(0) = 0.57 and P(l) = 0.43. The partitions 
5 2 (0, 0), 5 2 (0, 1), 5 2 (1, 0), 5 2 (1, 1) are also demonstrated in 
Figure HI a), and are given by 

5 2 (0,0) = (0,x tl ), 5 2 (0,1) = (x t i,x b ), 

S 2 (l,0) = (1*2,1), S 2 (l,l) = (x b ,x t2 ), (14) 

where x t i = 0.24 and x t2 = 0.86. The probability of 
occurrence of each of the 2-bit sequences can be calculated 
using ( fTTT i by integrating the asymptotic density distribution 
over the corresponding partitions, and is given by 

JM*i,t a )= / foo{x)dx (15) 

J S2(ii ,12) 

for h,i 2 = {0,1}. We have P 2 (0,0) = 0.35, P 2 (0, 1) = 0.22, 
P 2 (l, 0) = 0.23 and P 2 (l, 1) = 0.20 for the sample map. 

IV. Information Entropy as a Performance Metric 

In this section, we calculate the information entropy as the 
performance evaluation metric. The information entropy H for 
a single bit i is defined as fl9) 

H = -P(i = 0) log 2 P(i = 0) - P(i = 1) log 2 P(i = 1). 

(16) 

It can be shown that < H < 1. This equation implies that the 
information entropy of the bit-generation process is equal to 
H bits. H takes its maximum value of 1 bit if P(0) = P(l) = 
1/2, i.e., for this case the number of uncertain bits is equal to 
1. H is equal to if P(0) = or P(0) = 1. These two cases 
stand for the deterministic bit-generation process and there is 
no uncertainty or entropy in the process. For < P(0) < 1/2, 
the information entropy and the uncertainty will increase from 
the minimum of bit to the maximum of 1 bit. In this case, the 
likelihood of the occurrence of a 1 is more than the likelihood 
of the occurrence of a 0. This will lead to a decrease in the 
information entropy. 

The information entropy Hn of an iV-bit sequence is 
defined according to the probability of occurrence of each of 
the 2 N sequences, as given by 

Hn = - p N(k,— ,ijv-)logPjv(£i,...,ijv-)- 

u,...,i N e{o,i} 

(17) 

It can be shown that < Hn < N, i.e., the information 
entropy of an TV-bit sequence is at most equal to N bits. In 
other words, there are at most N uncertain bits in a sequence 
of N bits. 

We define Hn as the entropy of the iV-th generated bit, as 
given by 

fiN = Hn — Hn-i- (18) 
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Fig. 2. /ijy: the information entropy of the N-th generated bit. 

Hm is the information entropy of the generation of a bit, where 
the previous N — 1 generated bits are known. The relation 
between Hn and Hn can be written as 

N 

H N = Y J h k - (19) 

fc=i 

h n is a monotonically decreasing sequence of iV since the 
knowledge of N + 1 previous bits can decrease the entropy 
of the bit-generation more than the knowledge of N previous 
bits. It can also be shown that Hn converges to a fixed value 
h as N —> oo, given by 

h = lim hjq. (20) 

We propose h as a metric for the performance of truly random 
number generators. We can approximate h with Hn and since 
h,N is monotonically decreasing with N, we can decrease the 
approximation error as much as necessary by increasing N. 

We used the developed method to calculate the information 
entropy of the bits for the sample map. In Figure [2] Hn 
calculated by the developed method is demonstrated. As 
expected, both sequences are monotonically decreasing and 
both converge to the same point. An asymptotic entropy of 
more than 0.98 is achieved, which demonstrates a good quality 
of the generated bit sequence. 

h can also be utilized for design of an efficient post- 
processing unit. The degraded information entropy of the 



generated bit sequence, makes it necessary to decrease the 
bit-rate R of the sequence in order to obtain truly random 
bits. This is due to the fact that any deterministic processing 
performed on the sequence can not compensate for the lost 
information entropy. At least, the bit-rate R of the sequence 
has to be decreased by a factor of h to ensure that the 
information entropy of the resulting sequence can meet the 
threshold for being truly random, as given by 

R d = jR, (21) 
h 

where Rd is the decreased bit-rate. Note that special al- 
gorithms must be used to ensure that the generated binary 
sequence is truly random. 

V. Conclusion 

In this paper, we developed a methodology to calculate the 
information entropy of the bits generated from a discrete-time 
chaotic map. Information entropy is proposed as a metric 
for performance evaluation of the discrete-time chaos-based 
TRNGs. The developed metric enables the efficient design 
and optimization of TRNG circuits. The information entropy 
metric can also be used for the design of efficient post- 
processing units for the truly random number generators. 
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